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Description 

BACKGROUND OF THE INVENTION 

1 . FIELD OF THE INVENTION: 5 

[0001] The present invention relates to an apparatus 
for transmitting and receiving stream data which forms a 
three-dimensional virtual space represented by three- 
dimensional computer graphics (hereinafter, referred to 
as three<iimensional CG or 3DCG) based on a network 
such as the Internet. 

2. DESCRIPTION OF THE RELATED ART: 

[0002] In recent years, as the fields to which 3DCG is 
applicable, a virtual mall, electric commerce, and vari- 
ous home pages related thereto on the Internet such as 
a WWW (World Wide Web) have drawn attention. In 
particular, with the rapid development of the Internet, an 
environment is becoming available for easily handling 
3DCG with a relatively high quality such as games and 
movies at home. In the conventional WWW, a machine 
called a server such as a personal computer and a work 
station is connected to a plurality of machines called cli- 
ents such as personal computers. When a client 
requests, the client downloads data such as information 
of an image, a voice, a text, and an arrangement pro- 
vided by the server, if required. The data is recon- 
structed on the client side, whereby a user can obtain 
required information. A communication method based 
on a TCP/IP (Transmission Control/Internet Protocol) is 
adopted for communication between the server and the 
client. 

[0003] Conventionally, data provided by the server is 
mainly text data and image data. Recently, with the 
advancement in standardization of VRML (Virtual Real- 
ity Modeling Language), a browser of VRML, and the 
like, there is a tendency of transferring 3DCG data such 
as a shape and/or a scene. 

[0004] Hereinafter, the VRML will be briefly described. 
[0005] According to a conventional data format such 
as an image and a text as in HTML (Hyper Text Markup 
Language), a great amount of transfer time and transfer 
cost are required for transferring image data, in particu- 
lar, animation data. Therefore, in the current system, 
there is a limit to the network traffic. In contrast, accord- 
ing to conventional 3DCG, visual point information and 
light source information including configurations are all 
processed by using three-dimensional data obtained by 
CG (Computer Graphics) data by calculation. On the 
receiving side, image data is generated based on three- 
dimensional data, and it is displayed. 
[0006] With the advancement in CG technology, the 
quality of an image created by CG is rapidly improved. 
Therefore, it is becoming very efficient to transfer CG 
data as it is in view of a data amount. 
[0007] Typically, when CG data is used, the data can 



be compressed at a data compression ratio of 1/100 or 
less, compared with the case where comparable image 
data is transferred. Thus, there is a tendency of stand- 
ardizing a method for transferring 3DCG data through a 
network. As one solution, it is proposed to standardize 3 
DCG data called VRML (VRML Ver. 2.0). In VRML Ver. 
2.0, formats of data such as data having a configuration 
called a primitive, light source data, visual point data, 
and texture data, and a method for specifying a move- 
ment of a rigid body. 

[0008] On the other hand, there is an animation tech- 
nique for generating an image in real time, which has 
drawn attention recently in the CG field. By using an ani- 
mation technique in real time, it is attempted to repro- 
duce a real movement of a CG character particularly in 
CMs and movies. As an example, a complicated shape 
of a human is represented by a skeleton structure, and 
a motion amount of joints of a skeleton which change 
from moment to moment is defined, whereby a compli- 
cated motion can be naturally reproduced. 
[0009] However, according to the conventional 3 DCG 
modeling language on the Internet based on the VRML, 
a motion cannot be set in real time to a structure having 
a complicated shape such as a human. 
[0010] Furthermore, it is impossible to transmit and 
receive motion data of a CG character (e.g., human) 
which performs real motions in real time through a net- 
work with a narrow band such as a telephone line. 

SUMMARY OF THE INVENTION 

[001 1 ] A transmitter of the present invention includes 
a multi-dimensional stream data transmitting unit, for 
transmitting multi-dimensional stream data. 
[0012] In one embodiment of the present invention, 
the transmitter includes: a multidimensional stream 
data identifying unit for classifying input data into a plu- 
rality of kinds of motion data: and a multi-dimensional 
stream data compressing unit for compressing the plu- 
rality of kinds of motion data on a block basis. 
[0013] In another embodiment of the present inven- 
tion, the multi-dimensional stream data compressing 
unit changes a compression ratio in accordance with 
the kind of the motion data. 

[0014] In another embodiment of the present inven- 
tion, the multi-dimensional stream data compressing 
unit selects a compression method in accordance with 
the kind of the motion data. 

[0015] In another embodiment of the present inven- 
tion, the multi-dimensional stream data compressing 
unit generates a node sequence which is time-series 
data, composed of at least one motion data among the 
plurality of kinds of motion data during a predetermined 
period, and extracts an arbitrary node from the node 
sequence to compress the plurality of kinds of motion 
data on a block basis. 

[0016] In another embodiment of the present inven- 
tion, the multi-dimensional stream data transmitting unit 
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transmits interpolation information required for repro- 
ducing the extracted node and the node sequence. 
[0017] In another embodiment of the present inven- 
tion, the multi-dimensional stream data compressing 
unit reduces a high frequency component of the plurality 
of kinds of motion data to compress the plurality of kinds 
of motion data on a block basis. 
[0018] In another embodiment of the present inven- 
tion, the multi-dimensional stream data compressing 
unit subjects at least one motion data among the plural- 
ity of kinds of motion data to coordinate transformation 
to compress the plurality of kinds of motion data on a 
block basis. 

[0019] In another embodiment of the present inven- 
tion, the multi-dimensional stream data compressing 
unit transforms data represented by three-dimensional 
vector coordinates among the plurality of kinds of 
motion data to data represented by polar coordinates, 
thereby compressing the plurality of kinds of motion 
data on a block basis. 

[0020] In another embodiment of the present inven- 
tion, the multi-dimensional stream data includes header 
information or stream format describing data, describ- 
ing a kind of the multi-dimensional stream data to be 
transmitted, an order of the plurality of kinds of motion 
data to be transmitted, and a length of the plurality of 
kinds of motion data to be transmitted. 
[0021] In another embodiment of the present inven- 
tion, the multi-dimensional stream data has time infor- 
mation and space information of a plurality of 
dimensions. 

[0022] In another embodiment of the present inven- 
tion, the space information of a plurality of dimensions 
includes position information, rotation information, and 
scale information. 

[0023] In another embodiment of the present inven- 
tion, the multi-dimensional stream data capable of rep- 
resenting a CG scene includes data representing a 
position, rotation and a scale of the CG scene. 
[0024] In another embodiment of the present inven- 
tion, the multi-dimensional stream data capable of rep- 
resenting a CG scene is multi-dimensional animation 
stream data. 

[0025] In another embodiment of the present inven- 
tion, the multi-dimensional stream data has a data por- 
tion having a plurality of channels and a header portion 
describing a stream data format of the data portion and, 
each of the plurality of channels has a plurality of pieces 
of information regarding a three-dimensional CG char- 
acter at a certain time. 

[0026] In another embodiment of the present inven- 
tion, the three-dimensional CG character is capable of 
being represented by a hierarchical structure having at 
least one joint, and each of the plurality of motion data 
has at least one of data having a dimension of a position 
vector at a reference position of the three-dimensional 
CG character, data having a dimension of a direction 
vector representing a posture at the reference position, 



v data having a dimension of an angle vector of the at 
least one joint and a rotation angle around the angle 
vector, and stream data having a dimension of a rotation 
angle around a local coordinates axis of the at least one 
5 joint. 

[0027] In another embodiment of the present inven- 
tion, the multi-dimensional stream data is a packet hav- 
ing a header portion and a data portion, and the header 
portion represents a kind of the packet, an order of the 
10 packet, and a length of the data portion. 

[0028] In another embodiment of the present inven- 
tion, the above-mentioned transmitter includes a multi- 
dimensional stream data retaining unit for retaining data 
compressed by the multi -dimensional stream data com- 
15 pressing unit in a multi -hierarchical structure, wherein 
the transmitter selects data retained in a certain hierar- 
chical structure from the data retained in the multi-hier- 
archical structure, in accordance with a state of a 
transmission path through which the multi -dimensional 
20 stream data is to be transmitted and a reproduction 
speed at which a receiver receiving the multi-dimen- 
sional stream data reproduces the multi -dimensional 
stream data, and transmits the selected data. 
[0029] In another embodiment of the present inven- 
ts tion, the above-mentioned transmitter selects data at a 
hierarchy level, at which a reproducible data amount is 
the largest, from the data retained in the multi-hierarchi- 
cal structure, in accordance with a state of a transmis- 
sion path through which the multi-dimensional stream 
30 data is to be transmitted, and a reproduction speed at 
which the receiver receiving the multi -dimensional 
stream data reproduces the multi-dimensional stream 
data. 

[0030] In another embodiment of the present inven- 
35 tion, the above-mentioned transmitter further includes a 
multi-dimensional stream data retaining unit for retain- 
ing data compressed by the multi-dimensional stream 
data compressing unit in a file format of the multi-dimen- 
sional stream data. 
40 [0031] According to another aspect of the present 
invention, a transmitting method includes a multi-dimen- 
sional stream data transmitting step of transmitting 
multi -dimensional stream data. 

[0032] In one embodiment of the present invention, 
45 the above-mentioned transmitting method includes: a 
multi-dimensional stream data identifying step of classi- 
fying input data into a plurality of kinds of motion data; 
and a multi-dimensional stream data compressing step 
of compressing the plurality of kinds of motion data on a 
so block basis. 

[0033] In another embodiment of the present inven- 
tion, the multi-dimensional stream data compressing 
unit changes a compression ratio in accordance with 
the kind of the motion data. 
55 [0034] In another embodiment of the present inven- 
tion, the multi-dimensional stream data compressing 
unit selects a compression method in accordance with 
the kind of the motion data. 
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[0035] In another embodiment of the present inven- 
tion, in the mufti-dimensional stream data compressing 
step, a node sequence is generated, which is time- 
series data, composed of at least one motion data 
among the plurality of kinds of motion data during a pre- 
determined period, and extracts an arbitrary node from 
the node sequence to compress the plurality of Kinds of 
motion data on a block basis. 

[0036] In another embodiment of the present inven- 
tion, the above-mentioned transmitting method includes 
the step of transmitting interpolation information 
required for reproducing the extracted node and the 
node sequence. 

[0037] In another embodiment of the present inven- 
tion, in the multi-dimensional stream data compressing 
step, a high frequency component of the plurality of 
kinds of motion data is reduced to compress the plural- 
ity of kinds of motion data on a block basis. 
[0038] In another embodiment of the present inven- 
tion, in the multi-dimensional stream data compressing 
step, at least one motion data among the plurality of 
kinds of motion data is subjected to coordinate transfor- 
mation to compress the plurality of kinds of motion data 
on a block basts. 

[0039] In another embodiment of the present inven- 
tion, in the multi-dimensional stream data compressing 
step, data represented by three-dimensional vector 
coordinates among the plurality of kinds of motion data 
is transformed to data represented by polar coordinates, 
thereby compressing the plurality of kinds of motion 
data on a block basis. 

[0040] In another embodiment of the present inven- 
tion, the multi-dimensional stream data includes header 
information or stream format describing data, describ- 
ing a kind of the multi-dimensional stream data to be 
transmitted, an order of the plurality of kinds of motion 
data to be transmitted, and a length of the plurality of 
kinds of motion data to be transmitted. 
[0041] In another embodiment of the present inven- 
tion, the multi-dimensional stream data has time infor- 
mation and space information of a plurality of 
dimensions. 

[0042] In another embodiment of the present inven- 
tion, the space information of a plurality of dimensions 
includes position information, rotation information, and 
scale information. 

[0043] In another embodiment of the present inven- 
tion, the multi-dimensional stream data capable of rep- 
resenting a CG scene includes data representing a 
position, rotation and a scale of the CG scene. 
[0044] In another embodiment of the present inven- 
tion, the multi-dimensional stream data capable of rep- 
resenting a CG scene is multi-dimensional animation 
stream data. 

[0045] In another embodiment of the present inven- 
tion, the multi-dimensional stream data has a data por- 
tion having a plurality of channels, and a header portion 
describing a stream data format of the data portion, and 



each of the plurality of channels has a plurality of pieces 
of information regarding a three-dimensional CG char- 
acter at a certain time. 

[0046] In another embodiment of the present inven- 
5 tion, the three-dimensional CG character is capable of 
being represented by a hierarchical structure having at 
least one joint, and each of the plurality of motion data 
has at least one of data having a dimension of a position 
vector at a reference position of the three-dimensional 
10 CG character, data having a dimension of a direction 
vector representing a posture at the reference position, 
data having a dimension of an angle vector of the at 
least one joint and a rotation angle around the angle 
vector, and stream data having a dimension of a rotation 
is angle around a local coordinates axis of the plurality of 
joints. 

[0047] In another embodiment of the present inven- 
tion, the multi-dimensional stream data is a packet hav- 
ing a header portion and a data portion, and the header 

20 portion represents a kind of the packet, an order of the 
packet, and a length of the data portion. 
[0043] In another embodiment of the present inven- 
tion, the above-mentioned transmitting method includes 
a multi-dimensional stream data retaining step of retain- 

25 ing data compressed by the multi-dimensional stream 
data compressing unit in a multi -hierarchical structure, 
wherein the transmitting method selects data retained in 
a certain hierarchical structure from the data retained in 
the multi-hierarchical structure, in accordance with a 

30 state of a transmission path through which the multi- 
dimensional stream data is to be transmitted and a 
reproduction speed for reproducing the multi-dimen- 
sional stream data, and transmits the selected data. 
[0049] In another embodiment of the present inven- 

35 tion, data at a hierarchy level, at which a reproducible 
data amount is the largest, is selected from the data 
retained in the multi-hierarchical structure, in accord- 
ance with a state of a transmission path through which 
the multi-dimensional stream data is to be transmitted, 

40 and a reproduction speed at which the receiver receiv- 
ing the multi-dimensional stream data reproduces the 
multi-dimensional stream data. 

[0050] In another embodiment of the present inven- 
tion, the above-mentioned transmitting method further 
45 includes a multi-dimensional stream data retaining step 
of retaining data compressed in the multi-dimensional 
stream data compressing step in a file format of the 
multi-dimensional stream data. 

[0051] According to another aspect of the present 
so invention, a receiver includes a multi-dimensional 
stream data receiving unit for receiving multi-dimen- 
sional stream data. 

[0052] In one embodiment of the present invention, 
the receiver includes: a multi-dimensional stream data 
55 identifying unit for identifying a data content of the multi- 
dimensional stream data from one of header informa- 
tion and stream format describing information of the 
multi-dimensional stream data; a multi-buffering unit for 
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sequentially reading the multi-dimensional stream data 
into a plurality of multi-buffers on a block basis; and a 
multi-dimensional stream data decompressing unit for 
decompressing the multi-dimensional put in blocks on a 
block basis. 

[0053] In another embodiment of the present inven- 
tion, the multi-dimensional stream data identifying unit 
determines whether or not the received multi-dimen- 
sional stream data is a channel definition packet 
[0054] In another embodiment of the present inven- 
tion, the multi-dimensional stream data identifying unit 
determines whether or not the received multi-dimen- 
sional stream data is a data packet. 
[0055] In another embodiment of the present inven- 
tion, in the case where the mutti -dimensional stream 
data identifying unit determines that the received multi- 
dimensional stream data is a channel definition packet, 
the multi-dimensional stream data identifying unit iden- 
tifies a time stamp, a total number of channels, and a 
compression ratio identifier representing a compression 
ratio of channel data from the received multi-dimen- 
sional stream data. 

[0056] In another embodiment of the present inven- 
tion, in the case where the multi -dimensional stream 
data identifying unit determines that the received multi- 
dimensional stream data is a data packet, the multi- 
dimensional stream data identifying unit identifies a time 
stamp and a total number of channels from the received 
multi-dimensional stream data. 

[0057] In another embodiment of the present inven- 
tion, the received multi-dimensional stream data con- 
tains compressed motion data and compression 
information. 

[0058] In another embodiment of the present inven- 
tion, the multi-dimensional stream data decompressing 
unit decompresses the compressed motion data based 
on the compression information. 
[0059] in another embodiment of the present inven- 
tion, the multi-dimensional stream data decompressing 
unit determines a decompression ratio based on the 
compression information. 

[0060] In another embodiment of the present inven- 
tion, the multi -dimensional stream data decompressing 
unit determines a decompression method based on the 
compression information. 

[0061] In another embodiment of the present inven- 
tion, in the case where the compressed motion data is 
generated by extracting arbitrary motion data from a 
plurality of motion data corresponding to a predeter- 
mined period, the multi-dimensional stream data 
decompressing unit recovers a plurality of motion data 
corresponding to the predetermined period based the 
extracted motion data and the compression information. 
[0062] In another embodiment oi the present inven- 
tion, the compression information is interpolation infor- 
mation. 

[0063] In another embodiment of the present inven- 
tion, the mutti-dimensional stream data decompressing 



unit decompresses the compressed motion data by 
subjecting the compressed motion data to coordinate 
transformation based on the compression information. 
[0064] In another embodiment of the present inven- 

5 tion, in the case where the compressed motion data is 
represented by polar coordinates, the multi-dimensional 
stream data decompressing unit decompresses the 
compressed motion data by converting the compressed 
motion data represented by polar coordinates into data 

10 represented by three-dimensional vector coordinates. 
[0065] In another embodiment of the present inven- 
tion, the multi-dimensional stream data has time infor- 
mation and space information of a plurality of 
dimensions. 

is [0066] In another embodiment of the present inven- 
tion, the space information of a plurality of dimensions 
has position information, rotation information, and scale 
information. 

[0067] In another embodiment of the present inven- 
20 tion, the multi -dimensional stream data capable of rep- 
resenting a CG scene has data representing a position, 
a rotation, and a scale of the CG scene. 
[0068] In another embodiment of the present inven- 
tion, the multi-dimensional stream data capable of rep- 
25 resenting a CG scene is multi-dimensional animation 
stream data. 

[0069] In another embodiment of the present inven- 
tion, the multi-dimensional stream data has a data por- 
tion having a plurality of channels and a header portion 
30 describing a stream data format of the data portion, and 
each of the plurality of channels has a plurality of pieces 
of information regarding a three-dimensional CG char- 
acter at a certain time. 

[0070] In another embodiment of the present inven- 
35 tion, the three<limensional CG character is capable of 
being represented by a hierarchical structure having at 
least one joint, and each of the plurality of motion data 
has at least one of data having a dimension of a position 
vector at a reference position of the three-dimensional 
40 CG character, data having a dimension of a direction 
vector representing a posture at the reference position, 
data having a dimension of an angle vector of the at 
least one joint and a rotation angle around the angle 
vector, and stream data having a dimension of a rotation 
45 angle around a local coordinates axis of the plurality of 
joints. 

[0071 ] In another embodiment of the present inven- 
tion, the multi-dimensional stream data is a packet hav- 
ing a header portion and a data portion, and the header 

so portion represents a kind of the packet, an order of the 
packet, and a length of the data portion. 
[0072] In another embodiment of the present inven- 
tion, the receiver indicates a compression ratio of the 
multi-dimensional stream data to a transmitter for trans- 

55 mitting the multi-dimensional stream data, in accord- 
ance with at least one of a state of a transmission path 
through which the muiti -dimensional stream data is to 
be transmitted and a reproduction speed at which the 
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receiver reproduces the multi-dimensional stream data. 
[0073] According to another aspect of the present 
invention, a receiving method includes a multi-dimen- 
sional stream data receiving step of receiving multi- 
dimensional stream data. 

[0074] In one embodiment of the present invention, 
the above-mentioned receiving method includes: a 
multi-dimensional stream data identifying step of identi- 
fying a data content of the multi-dimensional stream 
data from one of header information and stream format 
describing information of the multi-dimensional stream 
data; a multi-buffering step of sequentially reading the 
multi-dimensional stream data into a plurality of multi- 
buffers on a block basis; and a multi-dimensional stream 
data decompressing step of decompressing the multi- 
dimensional put in blocks on a block basis. 
[0075] In one embodiment of the present invention, in 
the multi-dimensional stream data identifying step, it is 
determined whether or not the received multi-dimen- 
sional stream data is a channel definition packet. 
[0076] In another embodiment of the present inven- 
tion, in the multi-dimensional stream data identifying 
step, it is determined whether or not the received multi- 
dimensional stream data is a data packet. 
[0077] In another embodiment of the present inven- 
tion, in the case where the multi-dimensional stream 
data identifying unit determines that the received multi- 
dimensional stream data is a channel definition packet, 
in the multi-dimensional stream data identifying step, a 
time stamp, a total number of channels, and a compres- 
sion ratio identifier representing a compression ratio of 
channel data are identified from the received multi- 
dimensional stream data. 

[0078] In another embodiment of the present inven- 
tion, in the case where the multi-dimensional stream 
data identifying unit determines that the received multi- 
dimensional stream data is a data packet, in the multi- 
dimensional stream data identifying step, a time stamp 
and a total number of channels are identified from the 
received multi-dimensional stream data. 
[0079] In another embodiment of the present inven- 
tion, the received multi -dimensional stream data con- 
tains compressed motion data and compression 
information. 

[0080] In another embodiment of the present inven- 
tion, the mufti-dimensional stream data decompressing 
unit decompresses the compressed motion data based 
on the compression information. 

[0081] In another embodiment of the present inven- 
tion, in the multi-dimensional stream data decompress- 
ing step, a decompression ratio is determined based on 
the compression information. 

[0082] In another embodiment of the present inven- 
tion, in the multi-dimensional stream data decompress- 
ing step, a decompression method is determined based 
on the compression information. 
[0083] In another embodiment of the present inven- 
tion, in the case where the compressed motion data is 



generated by extracting arbitrary motion data from a 
plurality of motion data corresponding to a predeter- 
mined period, in the multidimensional stream data 
decompressing step, a plurality of motion data corre- 
5 sponding to the predetermined period are recovered 
based the extracted motion data and the compression 
information. 

[0084] In another embodiment of the present inven- 
tion, the compression information is interpolation infor- 
10 mation. 

[0085] In another embodiment of the present inven- 
tion, in the rnulti-dimensional stream data decompress- 
ing step, the compressed motion data is decompressed 
by subjecting the compressed motion data to coordinate 

15 transformation based on the compression information. 
[0086] In another embodiment of the present inven- 
tion, in the case where the compressed motion data is 
represented by polar coordinates, in the multi-dimen- 
sional stream data decompressing step, the com- 

20 pressed motion data is decompressed by converting the 
compressed motion data represented by polar coordi- 
nates into data represented by three-dimensional vector 
coordinates. 

[0087] In another embodiment of the present inven- 
ts tion, the multi-dimensional stream data has time infor- 
mation and space information of a plurality of 
dimensions. 

[0088] In another embodiment of the present inven- 
tion, the space information of a plurality of dimensions 
30 has position information, rotation information, and scale 
information. 

[0089] In another embodiment of the present inven- 
tion, the multidimensional stream data capable of rep- 
resenting a CG scene has data representing a position, 
35 a rotation, and a scale of the CG scene. 

[0090] In another embodiment of the present inven- 
tion, the multi-dimensional stream data capable of rep- 
resenting a CG scene is multidimensional animation 
stream data. 

40 [0091 ] In another embodiment of the present inven- 
tion, the multi-dimensional stream data has a data por- 
tion having a plurality of channels and a header portion 
describing a stream data format of the data portion and, 
each of the plurality of channels has a plurality of pieces 

45 of information regarding a three-dimensional CG char- 
acter at a certain time. 

[0092] In another embodiment of the present inven- 
tion, the three-dimensional CG character is capable of 
being represented by a hierarchical structure having at 

so least one joint, and each of the plurality of motion data 
has at least one of data having a dimension of a position 
vector at a reference position of the three-dimensional 
CG character, data having a dimension of a direction 
vector representing a posture at the reference position. 

55 data having a dimension of an angle vector of the at 
least one joint and a rotation angle around the angle 
vector, and stream data having a dimension of a rotation 
angle around a local coordinates axis of the at least one 
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joint 

[0093] In another embodiment of the present inven- 
tion, the multi-dimensional stream data is a packet hav- 
ing a header portion and a data portion, and the header 
portion represents a kind of the packet, an order of the 
packet, and a length of the data portion. 
[0094] In another embodiment of the present inven- 
tion, a compression ratio of the multi-dimensional 
stream data is indicated to a transmitter for transmitting 
the multi-dimensional stream data, in accordance with 
at least one of a state of a transmission path through 
which the multi-dimensional stream data is to be trans- 
mitted and a reproduction speed for reproducing the 
multi-dimensional stream data. 

[0095] In another embodiment of the present inven- 
tion, the multi-dimensional stream data contains motion 
data and voice data, the motion data has time stamp 
information, and the voice data has time stamp informa- 
tion, and the receiver reproduces the motion data and 
the voice data in synchronization, in accordance with 
the time stamp information of the motion data and the 
time stamp information of the voice data. 
[0096] In another embodiment of the present inven- 
tion, the multi -dimensional stream data contains motion 
data and voice data, the motion data has time stamp 
information, and the voice data has time stamp informa- 
tion, and the receiving method further comprises the 
step of reproducing the motion data and the voice data 
in synchronization, in accordance with the time stamp 
information of the motion data and the time stamp infor- 
mation of the voice data. 

[0097] According to another aspect of the present 
invention, a recording medium capable of being read by 
a computer is provided, in which a program for allowing 
the computer to execute the above-mentioned transmit- 
ting method is recorded. 

[0098] According to another aspect of the present 
invention, a recording medium capable of being read by 
a computer is provided, in which a program for allowing 
the computer to execute the above-mentioned receiving 
method is recorded. 

[0099] According to another aspect of the present 
invention, a recording medium capable of being read by 
a computer is provided, in which a program for allowing 
the computer to execute the above-mentioned receiving 
method is recorded. 

[0100] According to another aspect of the present 
invention, a communication system includes: a transmit- 
ter for transmitting multi -dimensional stream data; and a 
receiver for receiving the multi-dimensional stream 
data. 

[0101] In one embodiment of the present invention, 
the transmitter includes one of an artificial satellite, a 
ground wave broadcasting station, and a wire broad- 
casting station, and the receiver is one of a digital TV 
and a set top box. 

[0102] In the above-mentioned transmitter, the multi- 
dimensional stream data contains data capable of rep- 



resenting a CG scene. 

[0103] In the above-mentioned transmitting method, 
the multi-dimensional stream data contains data capa- 
ble of representing a CG scene. 
5 [0104] In the above-mentioned receiver, the multi- 
dimensional stream data contains data capable of rep- 
resenting a CG scene. 

[0105] In the above-mentioned receiving method, the 
multi-dimensional stream data contains data capable of 

10 representing a CG scene. 

[0106] In the above-mentioned communication sys- 
tem, the multi-dimensional stream data contains data 
capable of representing a CG scene. 
[0107] Thus, the invention described herein makes 

is possible the advantage of providing an apparatus for 
transmitting and receiving multi-dimensional stream 
data, which is capable of transmitting and receiving 
motion data of a CG character in real time. 
[0108] This and other advantages of the present 

20 invention will become apparent to those skilled in the art 
upon reading and understanding the following detailed 
description with reference to the accompanying figures. 

BRIEF DESCRIPTION OF THE DRAWINGS 

25 

[0109] 

Figure 1 shows an example of a CG scene dis- 
played by a display device. 

30 

Figure 2 shows a hierarchical structure represent- 
ing a CG scene shown in Figure 1 as an example of 
multi-dimensional stream data. 

35 Figure 3 shows a system which transmits and 
receives multi-dimensional stream data in Embodi- 
ment 1 . 

Figure 4 shows an apparatus for transmitting and 
40 receiving multi-dimensional stream data. 

Figure 5 shows an example of a skeleton structure 
of a three-dimensional CG character. 

45 Figure 6 shows an example of classification of input 
data. 

Figure 7 shows an example of a block in a time axis 
direction of three-dimensional CG character motion 
so data. 

Figure 8 shows a relationship between the three- 
dimensional vector coordinates and the polar coor- 
dinates. 

55 

Figure 9A shows 11 nodes extracted from 30 
frameyblock, and Figure 9B shows 6 nodes 
extracted from 15 frame/block. 
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Figure 10 shows an example of a multi-dimensional 
stream data format. 

Figure 11 shows an exemplary structure of multi- 
dimensional stream data in a time axis direction. 

Figure 12 shows an example of stream data output 
from a transmitter in the case where data input to 
the transmitter is 30 frame/block. 

Figure 13 shows reproduction of a motion stream 
and a voice stream in the case where a receiver is 
provided with a reproduction unit. 

Figure 14 illustrates that a plurality of receivers are 
connected to the transmitter. 

Figure 15 shows an example in which time stamps 
are inserted in a stream authoring unit. 

Figure 16 is a flow chart illustrating the operation of 
the transmitter. 

Figure 17 is a flow chart illustrating the operation of 
the receiver. 

DESCRIPTION OF THE PREFERRED EMBODI- 
MENTS 

[01 1 0] The present invention will be described by way 
of illustrative embodiments with reference to the draw- 
ings. 

Embodiment 1 

[0111] Hereinafter, Embodiment 1 will be described 
with reference to Figures 1 to 3. 
[0112] In Embodiment 1, multi-dimensional stream 
data is transmitted and/or received, whereby multi- 
dimensional stream data is reproduced with a stable 
quality at a high speed. Herein, the multi-dimensional 
stream data may contain data representing a CG scene. 
The data representing a CG scene contains geometric 
information such as object information, light source 
information, and viewing point information. 
[01 1 3] Figure 1 shows an example of a CG scene dis- 
played by a display device. Figure 1 shows that a table 
is positioned in a room surrounded by a wall. 
[0114] Figure 2 shows a hierarchical structure repre- 
senting a CG scene shown in Figure 1 as an example of 
multi -dimensional stream data. As shown in Figure 2, 
the wall and the table are branched from the room which 
is a root. If there is a cup on the table, the cup is 
branched from the table. The room, wall, and table 
respectively contain data for forming these objects. In 
an example shown in Figure 2, the room, wall and table 
respectively contain data such as a position, a rotation, 
and a scale. 



[01 1 5] Figure 3 shows a system 1 00 which transmits 
and receives multi-dimensional stream data in Embodi- 
ment 1 . 

[01 1 6] The system 1 00 shown in Figure 3 is provided 
s with a multi-dimensional stream data transmitting unit 

1 01 , and a multidimensional stream data receiving unit 

102. The multi-dimensional stream data transmitting 
unit 1 01 transmits multi -dimensional stream data in time 
series, and the multi-dimensional stream data receiving 

10 unit 1 02 receives multi-dimensional stream data in time 
series. The multi-dimensional stream data receiving unit 
102 may reproduce a CG scene based on received 
multi-dimensional stream data. 

[0117] Herein, the multi-dimensional stream data 

15 includes time information and space information of a 
plurality of dimensions, and time and space are multi- 
dimensional. The space information of a plurality of 
dimensions may include position information, rotation 
information, and scale information of a CG character. 

20 Alternatively, the multi-dimensional stream data may 
include data representing a position, a rotation and a 
scale of a CG character. Furthermore, the multi-dimen- 
sionat stream data may be multi-dimensional animation 
stream data representing animation of a CG character. 

25 [0118] A transmission path through which multi- 
dimensional stream data is transmitted is not limited to 
a telephone line or the like, and may be wireless. Fur- 
thermore, the multi -dimensional stream data receiving 
unit 102 may be a personal computer, a digital TV, or the 

30 like, as long as it is provided with a display device. 

[0119] In Embodiment 1, CG data is transmitted as 
stream data. Therefore, even when a part of stream 
data is lost, the original CG scene can be easily repro- 
duced. An example of a data structure of the multi- 

35 dimensional stream data will be described later. 

[01 20] In Embodiment 1 , the multi-dimensional stream 
data transmitting unit may include an artificial satellite, 
and the multi-dimensional stream data receiving unit 
may be a digital TV. 

40 [01 21 ] In Embodiment 1 , the multi-dimensional stream 
data transmitting unit may include an artificial satellite, 
and the multi-dimensional stream data receiving unit 
may be a set top box. 

[0122] The artificial satellite receives the multi-dimen- 
45 sional stream data from a ground station. 

[01 23] In Embodiment 1 , the multi-dimensional stream 
data transmitting unit may include a ground wave broad- 
casting station, and the multi -dimensional stream data 
receiving unit may be a digital TV. 
so [01 24] In Embodiment 1 , the multi-dimensional stream 
data transmitting unit may include a ground wave broad- 
cast ng station, and the multi-dimensional stream data 
receiving unit may be a set top box. 
[01 25] In Embodiment 1 . the multi-dimensional stream 
55 data transmitting unit may include a wire broadcasting 
station, and the multi-dimensional stream data receiving 
unit may be a digital TV. 

[01 26] In Embodiment 1 , the multi-dimensional stream 
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data transmitting unit may include a wire broadcasting 
station, and the multi-dimensional stream data receiving 
unit may be a set top box. 

[0127] A program for allowing a computer to execute 
the above-mentioned multi-dimensional stream data 
transmitting method and/or receiving method may be 
recorded in a recording medium capable of being read 
by a computer. 

Embodiment 2 

[0128] Hereinafter, Embodiment 2 will be described 
with reference to Figures 4 to 1 6. 
[0129] In Embodiment 2, data representing a CG 
scene is compressed based on a tree structure of CG, 
and multi-dimensional stream data is transmitted at a 
higher speed. 

[0130] Figure 4 shows an apparatus 10 for transmit- 
ting and receiving multi-dimensional stream data. A 
transmitter 1 transmits multi -dimensional stream data, 
and a receiver 2 receives the transmitted multi-dimen- 
sional stream data. 

[0131] Herein, the multi -dimensional stream data 
includes time information and space information of a 
plurality of dimensions, and time and space are multi- 
dimensional. The space information of a plurality of 
dimensions may include position information, rotation 
information, and scale information of a CG character. 
Alternatively, the multi-dimensional stream data may 
include data representing a position, a rotation and a 
scale of a CG character. Furthermore, the multi-dimen- 
sional stream data may be multi -dimensional animation 
stream data representing animation of a CG character. 
[0132] The apparatus 10 shown in Figure 4 includes a 
transmitter 1, a receiver 2, a multi -dimensional stream 
data storing unit 31, and a multi-dimensional stream 
data storing unit 32. 

[0133] The transmitter 1, and the receiver 2 are real- 
ized by a computer apparatus such as a work station, a 
personal computer, and a game machine. 
[01 34] The transmitter 1 may have a stream authoring 
unit (not shown), or the stream authoring unit may be 
provided outside the transmitter 1 . In accordance with 
the user's instruction, the steam authoring unit gener- 
ates three-dimensional CG stream data representing a 
three-dimensional virtual space. The operation of the 
stream authoring unit will be described later. 
[01 35] Furthermore, the transmitter 1 includes a multi- 
dimensional stream data identifying unit 11, a multi- 
dimensional stream data compressing unit 12, and a 
multi-dimensional stream data transmitting unit 13. The 
receiver 2 includes a multi-dimensional stream data 
identifying unit 21, a multi-buffering unit 22, a multi- 
dimensional stream data decompressing unit 23, and a 
multi-dimensional stream data receiving unit 24. 
[01 36] First, a series of processing in the transmitter 1 
will be described. 

[0137] An example of identifying multi-dimensional 



stream data in the transmitter 1 will be described. As an 
example of a multi-dimensional stream, the case of 
transmitting motion data of a three-dimensional CG 
character will be described. 

s [01 38] Figure 5 shows an example of a skeleton struc- 
ture of a three-dimensional CG character. 
[0139] The skeleton of the three-dimensional CG 
character shown in Figure 5 corresponds to a humanoid 
skeleton, which has 22 joints. The first joint represents a 

io root. 

[0140] The multi-dimensional stream data identifying 
unit 1 1 classifies a plurality of data input to the transmit- 
ter 1. 

[0141] figure 6 shows an example of classification of 

is input data. The multi -dimensional stream data identify- 
ing unit 11 classifies the input data into a plurality of 
kinds of motion data of the three-dimensional CG char- 
acter and voice data SS. A plurality of kinds of motion 
data of the three-dimensional CG character obtained by 

20 classifying the input data contain a position vector MS1 
of a reference position of the three-dimensional CG 
character in a three-dimensional space, a direction vec- 
tor MS2 representing the posture of the three-dimen- 
sional CG character at the reference position, and 

25 information MS3 of an angle vector at each joint of the 
skeleton and a rotation angle around the angle vector. 
[0142] In the case where the three-dimensional CG 
character has the above-mentioned skeleton, MS1 may 
be a position vector of the root, MS2 may be a direction 

30 vector and a rotation angle around the direction vector, 
and MS3 may be a rotation axis vector at each joint and 
a rotation angle around the rotation axis vector. A gen- 
eral expression of a hierarchical structure of a three- 
dimensional CG character is described in detail in Jap- 

35 anese Patent Application No. 9-100458 filed by the 
Applicant. Furthermore, the multi-dimensional stream 
data identifying unit 11 may classify the input data into 
data other than motion data and voice data SS. 
[0143] The multi-dimensional stream data compress- 

40 ing unit 1 2 in the transmitter 1 applies an individual com- 
pression method to the classified data MS1 to MS3 
based on the tree structure of CG and/or the kind of 
motion data to be classified, and compresses the data 
MS1 to MS3 on a block basis. For example, the multi - 

45 dimensional stream data compressing unit 12 changes 
the compression ratio depending upon the kind of 
motion data. 

[0144] A method for compressing multi -dimensional 
stream data by the multi -dimensional stream data com- 
so pressing unit 12 will be described with reference to Fig- 
ures 7 and 8. 

[0145] With respect to the data MS1 to MS3 of the 
three-dimensional CG character classified by the multi- 
dimensional stream data identifying unit 11, the multi- 
55 dimensional stream data compressing unit 12 puts 
motion data in one block during an arbitrary period (time 
t to time V) in a time axis direction, as shown in Figure 
7. Next, the multi-dimensional stream data compressing 
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unit 12 applies a different compression method, 
depending upon the kind ol the data W3S1 to MS3. 
[0146] A block method for compressing the data MS1 
to RflS3 on a block basis by the multidimensional 
stream data compressing unit 12 will be described. 
[0147] The position vector SWS1 at the reference posi- 
tion of the three-dimensional CG character has three 
floating points representing a position (x, y, z) in a three- 
axis direction. By reducing the precision of each value, 
data compression can be performed. 
[0148] For example, in the case where each compo- 
nent of the position (x, y, z) is represented by 4 bytes (32 
bits), the mufti-dimensional stream data compressing 
unit 12 performs data compression by representing 
each component of the position by 2 bytes (16 bits). 
[0149] When each component of the position is repre- 
sented by 2 bytes, three floating points can be 
expressed in 65536 stages (-32768 to 32767) in a deci- 
mal number. By expressing the floating points of 4 bytes 
in 65536 stages, data of 4 bytes can be compressed to 
data of 2 bytes. For example, the mutti -dimensional 
stream data compressing unit 12 may search for a max- 
imum value and a minimum value of input floating point 
data during a predetermined period, divided the differ- 
ence between the maximum value and the minimum 
value into 65536 stages, and express 4-byte data by 2 
bytes. 

[0150] In the above-mentioned example, data of 4 
bytes is expressed by 2 bytes. However, the multi- 
dimensional stream data compressing unit 12 may 
express a component of an N-byte position by 0 to (N-1 ) 
byte. 

[01 51 ] The direction vector RflS2 representing the pos- 
ture of the three-dimensional CG character contains 3 
components (three-dimensional vector) and the infor- 
mation ftflS3 of the angle vector at each joint and the 
rotation angle around the angle vector contains 4 com- 
ponents in total (3 components (three-dimensional vec- 
tor) representing a rotation axis in a local coordinate 
system at each hierarchical level of the three-dimen- 
sional CG character and one component representing 
the rotation angle). The multi -dimensional stream data 
compressing unit 12 normalizes (sets the magnitude of 
a vector at 1) the three-dimensional vector representing 
the rotation axis, i.e., P (X, Y, Z) in Figure 8, and con- 
verts rectangular coordinates into polar coordinates, 
thereby expressing three components (X, Y, Z) by two 
components ( x , 6 ). More specifically, the direction vec- 
tor WJS2 can be compressed from 3 components to 2 
components in a space axis direction. The information 
MS3 can be compressed from 4 components to 3 com- 
ponents in a space axis direction. 
[0152] Furthermore, the multi-dimensional stream 
data compressing unit 12 may compress the position 
vector RflS1 at the reference position of the three-dimen- 
sional CG character and the angle vector at each joint 
and the rotation angle SV1S3 around the angle vector in a 
time axis direction. For example, the mufti-dimensional 



stream data compressing unit 12 generates a node 
sequence from a plurality of kinds of motion data (e.g., 
at least one of the position vector AflS1 and the informa- 
tion RflS3), and extracts an arbitrary node from the node 
5 sequence, thereby compressing a plurality of kinds of 
motion data on a block basis. 

[0153] Figure 9A shows 1 1 nodes extracted from 30 
frame/block. Figure 9B shows 6 nodes extracted from 
15 frame/block. 

10 [01 54] The nodes shown in Figures 9A and 9B repre- 
sent at least one of the position vector MSI and the 
information MS3. A node column represents at least 
one of the position vector RflSl and the information ftflS3 
in time sequence. 

is [0155] As shown in Figure 9A, in the case where the 
nodes in one block are denoted with Nos. 0 to 29 in 30 
frame/block, 11 nodes of Nos. 0, 2, 5, 8, 11, 14, 18, 21, 
24, 27, and 29 are extracted. Only the extracted nodes 
are output from the multi-dimensional stream data com- 

20 pressing unit 12, and the other nodes are not output 
from the multi-dimensional stream data compressing 
unit 12. 

[0156] Furthermore, as shown in Figure 9B, in the 
case where the nodes in one block are denoted with 
25 Nos. 0 to 14 in 15 frame/block, 6 nodes of Nos. 0, 2, 5. 
9. 12, and 14 are extracted. Only the extracted nodes 
are output from the multi-dimensional stream data com- 
pressing unit 12, and the other nodes are not output 
from the multi-dimensional stream data compressing 
•30 unit 12. 

[0157] In the case where a node extracted from the 
multi-dimensionai stream compressing unit 12 is output, 
the multi-dimensional stream compressing unit 12 may 
output interpolation information required for recovering 

35 an original node sequence from the extracted node. 
[01 58] The number of nodes to be extracted per block 
is not limited to 1 1 or 6. The number of nodes to be 
extracted is appropriately determined by a user or the 
like. When it is desired to enhance an image quality of a 

40 CG scene to be transmitted, the number of nodes to be 
extracted is increased. When a CG scene is transmitted 
at a high speed, the number of nodes to be extracted 
should be decreased. 

[0159] The extracted nodes are subjected to data 
45 interpolation by three-dimensional spline interpolation 
in the receiver 2. Other interpolation methods such as a 
linear interpolation may be used in place of the three- 
dimensional interpolation. 

[0160] The multi -dimensional stream data compress- 
so ing unit 12 may reduce at least one high frequency com- 
ponent of a plurality of kinds of motion data, and 
compress the motion data on a block basis. 
[01 61 ] Next, the multi -dimensional stream data storing 
unit 31 will be described in detail. 
55 [0162] Stream data ftflSCI to MSC3, and SSC com- 
pressed on a block basis are stored, as multi-dimen- 
sional stream data, in a storage device in an ordinary 
computer apparatus in the multi-dimensional stream 
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data storing unit 31 . 

[0163] Hereinafter, a format of multi-dimensional 
stream data stored in the multi-dimensional stream data 
storing unit 31 will be described in detail. 
[0164] The multi-dimensional stream data compress- 
ing unit 1 2 puts compressed motion data in a block, and 
adds header information for transmitting multi-dimen- 
sional stream data to the block. The block and the 
header information are stored in the multi-dimensional 
stream data storing unit 31 . 

[0165] Figure 10 shows an example of a format of 
multi-dimensional stream data stored in the multi- 
dimensional stream data storing unit 31 . 
[0166] The multi-dimensional stream data is com- 
posed of packets. Each packet has a header portion 
and a data portion. Depending upon the kind of stream 
data, one stream data will be referred to as a channel, 
and each channel is provided with a uniquely deter- 
mined channel No. (There is no limit to the order of Nos. 
to be provided to the channels). 

[0167] The multi-dimensional stream data stored in 
the multi-dimensional stream data storing unit 31 
includes a channel definition packet and a data packet, 
as shown in Figure 10. The channel definition packet 
refers to a stream defining the channels in the data 
packet which follows after the channel definition packet 
is transmitted. The channel definition packet and the 
data packet respectively have a header portion and a 
data portion. 

[0168] The header portion of the channel definition 
packet includes, for example, a packet identifier which 
identifies itself as the channel definition packet, a time 
stamp which represents a time based on a certain refer- 
ence time when the channel definition packet is gener- 
ated, the total number TC of channels through which 
data can be transmitted in the channel definition packet, 
and a compression ratio identifier which represents the 
number of nodes when the nodes are extracted under 
compression on a block basis of the stream data. The 
data portion of the channel definition packet includes 
the total number TC of channels. Each channel of the 
channel definition includes a channel identifier which 
represents a channel No., a data size of the channel, a 
channel name, and a channel type which represents if 
the channel is a position vector or a rotation angle vec- 
tor. 

[0169] As described above, the data packet includes 
a header portion and a data portion. 
[0170] The header portion of the data packet includes, 
for example, a packet identifier which identifies itself as 
a data packet, a time stamp which represents a time 
based on a certain reference time when the channel 
definition packet is generated, and the total number DC 
of channels (TC § DC) through which data can be trans- 
mitted in the data packet. 

[0171] The data portion of the data packet includes 
the total number DC of channels, and each channel 
includes a channel identifier which represents a channel 



No., channel data and compression ratio identifier (e.g., 
the number of extracted notes, etc.). It is noted that the 
number of compression identifiers is equal to that of the 
channel data. The data packet is equal to one block of 

5 multi-dimensional steam data. 

[01 72] Figure 1 1 shows an exemplary structure of the 
channel definition packet and the data packet. When the 
channel definition packet is first transmitted, only the 
data packets may be transmitted thereafter. In the case 

10 where it is desired to change the channel definition in 
the middle of a stream, the channel definition can be 
easily changed by transmitting the channel definition 
packet in the middle. 

[0173] The multi-dimensional stream data is not lim- 

15 ited to the packet shown in Figure 1 0. For example, the 
multidimensional stream data may have header infor- 
mation or stream format describing data, describing the 
kind of multi-dimensional stream data, the order of a 
plurality of motion data, and the length of a plurality of 

20 motion data. 

[01 74] Next, the multi-dimensional stream data storing 
unit 31 stores a plurality of stream data having different 
compression ratios. In the case where an excessive or 
insufficient amount of data flows through a transmission 

25 path between the transmitter 1 and the receiver 2, in 
order to effectively utilize the transmission path, the 
transmitter 1 selects stream data with an optimum com- 
pression ratio, in accordance with at least one of the 
state of the transmission path, the request from the 

30 receiver 2, and the request from a user. This will be 
described below. To store a plurality of stream data hav- 
ing different compression ratios will be referred to as a 
multi-hierarchical storage method. 
[01 75] The transmitter 1 generates stream data which 

35 is compressed at a certain compression ratio on a block 
basis, and generates some kinds of stream data which 
are compressed at a different compression ratio from 
that of the compressed stream data. The stream data 
compressed at a different compression ratio are stored 

40 in the multi -dimensional stream data storing unit 31 . 
[0176] Figure 12 shows an example of stream data 
output from the transmitter 1 in the case where data 
input to the transmitter 1 is 30 frame/block. 
[0177] The highest hierarchical level shown in Figure 

45 12 refers to stream data obtained by outputting input 
data in 30 frame/block as one block without compress- 
ing it. The second hierarchical level refers to stream 
data obtained by compressing input data in 30 
frame/block into data in 1 1 frame/block. The third hierar- 

so chical level refers to stream data obtained by thinning 
out every other input data in 30 frame/block, thereby 
compressing the input data into data in 15 frame/block 
which is a half data amount. The lowest hierarchical 
level refers to stream data obtained by compressing 

55 stream data in 1 5 frame/block into data in 6 frame/block. 
Furthermore, the stream data at the second hierarchical 
level and the stream data at the lowest hierarchical level 
are complemented by three-dimensional spline interpo- 
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lation. 

[0178] Multi-hierarchical stream data is stored in the 
multidimensional stream data storing unit 31 , whereby 
stream data containing an information amount in 
accordance with the state of the transmission path can s 
be transmitted. 

[0179] Furthermore, according to the multi-hierarchi- 
cal storage method, a compression ratio of stream data 
transmitted from the transmitter can be controlled to be 
optimum so as to be consistent with the ability of a con- 
trol circuit (e.g., computer) included in the receiver 2. 
[01 80] Next, the data transmission between the trans- 
mitter 1 and the receiver 2 will be described. 
[0181 ] In the case where there is an instruction from a 
control portion of the transmitter 1 or there is a request 
from the receiver 2, the transmitter 1 transmits multi- 
dimensional stream data stored in a storage device in a 
computer apparatus, i.e., the multi-dimensional stream 
data storing unit 31 . A medium for data transmission in 
this case is assumed to be a bi-directional communica- 
tion system, i.e., the Internet. However, data can be sim- 
ilarly transmitted and received via a broadcasting 
system such as satellite broadcasting and ground wave 
broadcasting. 

[0182] Hereinafter, a series of processing of the 
receiver 2 will be described. 

[0183] The multidimensional stream data receiving 
unit 24 receives multi-dimensional stream data transmit- 
ted from the transmitter 1 . 

[0184] Next, identification of mutti-dimensional stream 
data in the receiver 2 will be exemplified. The case 
where multi-dimensional stream data of a three-dimen- 
sional CG character is received will be described as an 
example of a multi-dimensional stream. The multi- 
dimensional stream data includes the above-mentioned 
compressed motion data in a predetermined stream for- 
mat. The motion data included in the multidimensional 
stream data may not be compressed. 
[0185] First, the multi-dimensional stream data receiv- 
ing unit 24 in the receiver 2 receives multi-dimensional 
stream data GWSC1 to MSC3 transmitted from the trans- 
mitter 1 . In the case where the transmitted multi-dimen- 
sional stream data COTSC1 to W3SC3 has a format as 
shown in Figure 10, the mutti-dimensional stream data 
identifying unit 21 first receives a packet, identifier and 
analyzes it, thereby determining if the received stream 
data is a channel definition packet or a data packet. 
[0186] In the case where the multidimensional stream 
data identifying unit 21 determines the transmitted 
multi-dimensional stream data RASC1 to RASC3 to be a 
channel definition packet, the multi-dimensional stream 
data identifying unit 21 then receives a time stamp, the 
total number of channels TC, and a compression ratio 
identifier, and analyzes them. Subsequently, the multi- 
dimensional stream data identifying unit 21 receives a 
data portion, i.e., channel identifiers, channel data 
sizes, channel names, and channel types respectively 
by the total number of channels TC, and analyze them. 



If the mutti-dimensional stream data identifying unit 21 
fails in data analysis, the receiver 2 requests the trans- 
mitter 1 to re-transmit data so as to obtain data again. 
Alternatively, the receiver 2 forcefully terminates a con- 
nection of the transmitter 1 and the receiver 2. Herein, 
the compression ratio identifier means compression 
information such as a compression method and a com- 
pression ratio used in the compression method. 
[0187] In the case where the multi-dimensional stream 
data identifying unit 21 determines that the transmitted 
multi-dimensional stream data MSC1 to MSC3 to be a 
data packet, the multi-dimensional stream data identify- 
ing unit 21 receives a time stamp, and the total number 
of channels «DC, and analyzes them. Then, the multi- 
dimensional stream data identifying unit 21 receives a 
data portion, i.e., a channel identifier, channel data, and 
a compression ratio identifier, and analyzes them. This 
is repeated by the total number of channels DC. The 
multi-dimensional stream data identifying unit 21 
assumes that only the position vector at the reference 
position and the direction vector representing a posture 
among motion data of a three-dimensional CG charac- 
ter have special channel names, and distinguishes 
these vectors from each other based on the channel 
names. 

[01 88] In the case where the multi-dimensional stream 
data identifying unit 21 fails in data analysis of the data 
packet, the receiver 2 requests the transmitter 1 to re- 
transmit data so as to obtain the data again. Alterna- 
tively, the receiver 2 forcefully terminates a connection 
of the transmitter 1 and the receiver 2. Alternatively, the 
receiver 2 discards the data packet whose analysis has 
failed, and receives a subsequent packet, whereby the 
connection can be continued. 

[0189] Next, the multi-buffering unit 22 in the receiver 
2 will be described. 

[0190] In the multi-buffering unit 22 in the receiver 2, 
multi-dimensional stream data divided into blocks is 
buffered on a block basis at a multiplicity which is con- 
sistent with processing performance or an execution 
speed of a computer having the receiver 2. More specif- 
ically, in the case where the processing performance 
and execution speed of the computer are large, multi- 
plicity of buffering is set to be large. In the case where 
the processing performance and execution speed of the 
computer are small, multiplicity of buffering is set to be 
small. In this case, data storage for buffering is per- 
formed by the multi-dimensional stream data storing 
unit 32. The multi-buffering will be described in more 
detail together with the later description of simultaneous 
reproduction. 

[0191] Next, the multi-dimensional stream data 
decompressing unit 23 in the receiver 2 will be 
described. 

[0192] In the multi-dimensional stream data decom- 
pressing unit 23 in the receiver 2, compressed stream 
data EWSC1 to RASC3 for one block of the data packet are 
decompressed by a different method depending upon 
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compression information represented by the compres- 
sion identifier. For example, the multi-dimensional 
stream data decompressing unit 23 may determine a 
decompression method based on compression informa- 
tion. The multi-dimensional stream data decompressing 
unit 23 may determine a decompression ratio for 
decompressing compressed motion data based on 
compression information. 

[0193] In the case where compression in a time axis 
direction in one block and compression for decreasing 
precision of a value of each component (e.g.. 4 bytes of 
data is compressed to 2 bytes of data) are performed in 
the position vector data MSC1 at a reference position of 
a three-dimensional CG character, the multi-dimen- 
sional stream data decompressing unit 23 first performs 
decompression (2 bytes of data is decompressed to 4 
bytes of data by reverse conversion of compression 
conversion) with respect to the compression for 
decreasing precision of a value of each component. 
Then, with respect to the compression in a time axis 
direction, the multi -dimensional stream data decom- 
pressing unit 23 performs decompression of one block 
of stream data by three-dimensional spline interpola- 
tion, linear interpolation, or the like, using a compres- 
sion identifier in the header portion of the channel 
definition packet. 

[0194] The multi-dimensional stream data decom- 
pressing unit 23 converts the direction vector data 
MSC2 from the polar coordinates to the three-dimen- 
sional vector coordinates. For example, in the case 
where the direction vector data MSC2 representing the 
posture of a three-dimensional CG character has two 
polar coordinates components representing a rotation 
axis and is received as stream data, the multi-dimen- 
sional stream data decompressing unit 23 may convert 
the position vector data MSC2 from the polar coordi- 
nates to the three-dimensional vector coordinates to 
obtain a direction vector representing a rotation axis 
with a magnitude 1 . 

[0195] In the angle vector at each joint of a three- 
dimensional CG character and rotation angle around 
the angle vector MSC3, in the same way as in the con- 
version from the polar coordinates in the direction vector 
representing the posture to the three-dimensional vec- 
tor coordinates, the multi-dimensional stream data 
decompressing unit 23 first obtains three three-dimen- 
sional vector coordinates from two polar coordinates. 
Then, in the same way as in the decompression of 
stream data in a time axis direction in the position vector 
MSC1 at the reference position, the multi-dimensional 
stream data decompressing unit 23 performs decom- 
pression of one block of stream data by three-dimen- 
sional spline interpolation, linear interpolation, or the 
like, using a compression ratio identifier in a header por- 
tion of the channel definition packet. 
[0196] After the decompression of multi-dimensional 
stream data is completed in the multi-dimensional 
stream. data decompressing unit 23, the multi-dimen- 



sional stream data storing unit 32 stores the decom- 
pressed multi -dimensional stream data. In the case 
where a three-dimensional CG reproduction unit pro- 
vided outside requests stream data, stream data is sup- 

5 plied from the multi-dimensional stream data storing 
unit 32, whereby the receiver 2 can reproduce three- 
dimensional CG scene in real time. 
[0197] Multi-buffering and synchronous reproduction 
of multi-dimensional stream data and a method for 

10 reproducing data in real time in the receiver 2 will be 
described in detail. 

[01 98] Hereinafter, the case where one motion stream 
data and one voice stream data of a three-dimensional 
CG character are buffered when the receiver 2 is pro- 
15 vided with a reproduction unit (not shown) will be 
described. 

[0199] Figure 13 shows reproduction of a motion 
stream and a voice stream in the case where the 
receiver 2 is provided with a reproduction unit. A repro- 

20 duction process and a buffering process are performed 
with respect to the motion stream and the voice stream. 
[0200] According to the reproduction process for 
reproducing a motion stream during one block of period 
(time t to time f). a reproduction unit reproduces the 

25 motion of a three-dimensional CG character from buff- 
ered motion stream data, in accordance with time 
stamps of the motion stream data. According to the buff- 
ering process of a motion stream, the receiver 2 
receives motion stream data, for example, in a block fol- 

30 lowing the block to which the motion stream data which 
is currently being reproduced belongs, i.e., in a block 
corresponding to a period (time f to time t"). The multi- 
dimensional buffering unit 22 buffers the received 
motion stream data. The multi -dimensional stream data 

35 identifying unit 21 analyzes motion stream data, and the 
multi -dimensional stream data decompressing unit 23 
decompresses motion stream data. In order to prepare 
for the reproduction of the motion stream data (data 
reproduction in the subsequent block), the multi-dimen- 

40 sional buffering unit 22 buffers the decompressed 
motion stream data again. 

[0201] Voice stream data is also subjected to the 
above-mentioned reproduction process and buffering 
process. Furthermore, even in the case where multiplie- 
rs ity of buffering is increased, multi-buffering can be real- 
ized by a similar operation. 

[0202] In the case where the transmission speed of 
stream data from the receiver 2 to the reproduction unit 
which reproduces three-dimensional CG data is higher 

so than the buffering speed, i.e., in the case where stream 
data cannot be received and decompressed before 
being transmitted to the reproduction unit, the receiver 2 
requests the transmitter 1 of compressed multi-dimen- 
sional stream data having a low data amount and an 

55 optimum compression ratio for data reproduction, and 
the receiver 2 receives data, whereby real time data 
reproduction can be maintained. 
[0203] A minimum structure of an apparatus for trans- 
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mitting and receiving mutti-dimensional stream data 
(i.e., the case where one receiver 2 is connected to one 
transmitter 1) has been described. However, as shown 
in Figure 14, a plurality of receivers 2 may be connected 
to one transmitter 1 via a transmission portion. 
[0204] As described above, the transmitter 1 may be 
provided with a stream authoring unit (not shown). 
Hereinafter, the basic operation of the stream authoring 
unit mentioned with reference to Figure 4 will be briefly 
described. The stream authoring unit cuts motion 
stream data and voice stream data of a three-dimen- 
sional CG character having an arbitrary data length into 
an arbitrary length required for synchronization. The 
data length of the motion stream data is determined 
based on the duration of an operation, and the data 
length of the voice stream data is determined based on 
the reproduction time of a voice. As shown in Figure 1 5, 
the stream authoring unit gives time stamp information 
required for synchronization to the motion stream and 
the voice stream. Furthermore, the stream authoring 
unit connects the motion stream to the voice stream, if 
required, and generates a new stream. The stream 
authoring unit registers the stream data thus edited in 
the transmitter 1 as input data. 

[0205] In the case where the receiver 2 shown in Fig- 
ure 4 has a reproduction unit, the motion stream is pref- 
erably reproduced in synchronization with the 
reproduction of the voice stream, based on time stamp 
information of the motion stream and time stamp infor- 
mation of the voice stream. Furthermore, the voice 
stream may be reproduced in synchronization with the 
reproduction of the motion stream, based on each time 
stamp. 

[0206] Hereinafter, the basic operation of the appara- 
tus for transmitting and receiving multi-dimensional 
stream data in the present embodiment will be 
described with reference to Figures 16 and 17. 
[0207] Figure 16 is a flow chart illustrating the opera- 
tion of the transmitter 1 , and Figure 1 7 is a flow chart 
illustrating the operation of the receiver 2. 
[0208] First, an exemplary basic operation of the 
transmitter 1 will be described. 

[0209] At Step S1 1 in Figure 16, stream data ftfiSI to 
RflS3 and SS created in the external stream data author- 
ing unit are input to the transmitter 1 . 
[0210] At Step S12, the transmitter 1 identifies the 
stream data RfiS1 to SWS3 and SS input at Step S1 1 , and 
compresses the stream data to obtain compressed 
stream data RASC1 to EWSC3 and SSC. The transmitter 
1 generates and stores multi-dimensional stream data 
packets. 

[021 1 ] At Step S1 3, the transmitter 1 transmits a chan- 
nel definition packet to the receiver 2. 
[0212] At Step S14, the transmitter 1 determines 
whether or not the receiver 2 requests that the channel 
definition should be changed. In the case where the 
channel definition should be changed, the process 
returns to Step S13. 



[021 3] At Step S1 5, the transmitter 1 transmits a data 
packet to the receiver 2. 

[0214] At Step S16, the transmitter 1 determines 
whether or not the amount of data which flows through 

5 a transmission path is excessive or insufficient. In the 
case where the amount data to be transmitted exceeds 
an allowable amount in the transmission path, or in the 
case where the amount of data to be transmitted is 
below the allowable amount, or in the case where the 

io receiver 2 requests that data to be transmitted should 
be changed, the process proceeds to Step S17. In the 
case where the transmitter 1 determines that the 
amount of data is not excessive or insufficient, the proc- 
ess proceeds to Step S14. 

is [0215] At Step S17, the transmitter 1 selects stream 
data with an optimum compression ratio consistent with 
the state of the transmission path, or stream data with 
an optimum compression ratio in accordance with a 
request from the receiver 2. Then, the process returns 

20 to Step S1 4. 

[0216] Next, an example of the operation of the 
receiver 2 will be described. 

[0217] At Step S21 in Figure 17, the receiver 2 
receives the multi-dimensional stream data packet from 

25 the transmitter 1 . 

[021 8] At Step S22, the receiver 2 determines whether 
or not the stream data packet received at Step 21 is a 
channel definition packet. In the case where the 
received packet is a channel definition packet, the proc- 

30 ess returns to Step S21 . 

[0219] At Step S23, the receiver 2 analyzes the data 
packet and data. 

[0220] At Step S24, the receiver 2 determines whether 
or not data is present in a buffer. In the case where data 

35 is not present, the process returns to Step S21 . 

[0221 ] At Step S25, the receiver 2 decompresses one 
block of stream data present in the buffer by the number 
of kinds of required stream data. 
[0222] At Step S26, the receiver 2 determines whether 

40 or not one block of data has been decompressed at 
Step S25. In the case where one block of data has been 
decompressed, the receiver 2 transmits the decom- 
pressed data to a reproduction unit. That is, the process 
proceeds to Step 27. In the case where one block of 

45 data has not been decompressed, the process pro- 
ceeds to Step S28. 

[0223] At Step $27, the receiver 2 transmits the 
decompressed stream data to a reproduction unit. At 
Step S27, the receiver 2 determines whether or not a 

so final flag has been generated by a user. In the case 
where no final flag has been generated, the process 
returns to Step S21 . In the case where a final flag has 
been generated, the receiver 2 terminates its operation. 
[0224] In Embodiment 2, the transmitter may include 

55 an artificial satellite, and the receiver may be a digital 
TV. 

[0225] In Embodiment 2, the transmitter may include 
an artificial satellite, and the receiver may be a set top 
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box. 

[0226] The artificial satellite receives the multi-dimen- 
sional stream data from a ground station. 
[0227] In Embodiment 2, the transmitter may include 
a ground wave broadcasting station, and the receiver 
may be a digital TV. 

[0228] In Embodiment 2, the transmitter may include 
a ground wave broadcasting station, and the receiver 
may be a set top box. 

[0229] In Embodiment 2, the transmitter may include 
a wire broadcasting station, and the receiver may be a 
digital TV. 

[0230] In Embodiment 2, the transmitter may include 
a wire broadcasting station, and the receiver may be a 
set top box. 

[0231] A program for allowing a computer to execute 
the above-mentioned multi-dimensional stream data 
transmitting method and/or receiving method may be 
recorded in a recording medium capable of being read 
by a computer. 

[0232] According to the present invention, motion 
data, voice data, and the like of a three-dimensional CG 
character are separately compressed depending upon 
the kind, and the compressed data is transferred as a 
stream in real time, while information required for repro- 
ducing each data in synchronization is simultaneously 
transferred. Thus, a smooth motion of the three-dimen- 
sional CG character and sound information in synchro- 
nization with the motion can be interactively transmitted 
and received in accordance with a user's request in a 
transmitting/receiving system based on a network, and 
the amount of data to be transferred can be substan- 
tially reduced. 

[0233] Various other modifications will be apparent to 
and can be readily made by those skilled in the art with- 
out departing from the scope and spirit of this invention. 
Accordingly, it is not intended that the scope of the 
claims appended hereto be limited to the description as 
set forth herein, but rather that the claims be broadly 
construed. 

Claims 

1. A transmitter comprising a multi-dimensional 
stream data transmitting unit, for transmitting multi- 
dimensional stream data. 

2. A transmitter according to claim 1 , comprising: 

a multi -dimensional stream data identifying unit 
for classifying input data into a plurality of kinds 
of motion data; and 

a multi-dimensional stream data compressing 
unit for compressing the plurality of kinds of 
motion data on a block basis. 

3. A transmitter according to claim 2, wherein the 
multi-dimensional stream data compressing unit 



changes a compression ratio in accordance with 
the kind of the motion data. 

4. A transmitter according to claim 2, wherein the 
s multi-dimensional stream data compressing unit 

selects a compression method in accordance with 
the kind of the motion data. . 

5. A transmitter according to claim 2, wherein the 
w multi-dimensional stream data compressing unit 

generates a node sequence which is time-series 
data, composed of at least one motion data among 
the plurality of kinds of motion data during a prede- 
termined period, and extracts an arbitrary node 
is from the node sequence to compress the plurality 
of kinds of motion data on a block basis. 

6. A transmitter according to claim 5, wherein the 
multi-dimensional stream data transmitting unit 

20 transmits interpolation information required for 
reproducing the extracted node and the node 
sequence. 

7. A transmitter according to claim 2 ( wherein the 
25 multi-dimensional stream data compressing unit 

reduces a high frequency component of the plural- 
ity of kinds of motion data to compress the plurality 
of kinds of motion data on a block basis. 

30 8. A transmitter according to claim 2, wherein the 
multi-dimensional stream data compressing unit 
subjects at least one motion data among the plural- 
ity of kinds of motion data to coordinate transforma- 
tion to compress the plurality of kinds of motion 

35 data on a block basis. 

9. A transmitter according to claim 2, wherein the 
multi-dimensional stream data compressing unit 
transforms data represented by three-dimensional 
40 vector coordinates among the plurality of kinds of 
motion data to data represented by polar coordi- 
nates, thereby compressing the plurality of kinds of 
motion data on a block basis. 

45 10. A transmitter according to claim 1, wherein the 
multi -dimensional stream data includes header 
information or stream format describing data, 
describing a kind of the multi-dimensional stream 
data to be transmitted, an order of the plurality of 

so kinds of motion data to be transmitted, and a length 
of the plurality of kinds of motion data to be trans- 
mitted. 

11. A transmitter according to claim 1. wherein the 
55 multi-dimensional stream data has time information 

and space information of a plurality of dimensions. 

12. A transmitter according to claim 11, wherein the 
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space information of a plurality of dimensions 
includes position information, rotation information, 
and scale information. 

13. A transmitter according to daim 1, wherein the s 
multi-dimensional stream data capable of repre- 
senting a CG scene includes data representing a 
position, rotation and a scale of the CG scene. 

14. A transmitter according to claim 1, wherein the 10 
multi-dimensional stream data capable of repre- 
senting a CG scene is multi-dimensional animation 
stream data. 

15. A transmitter according to claim 1, wherein the is 
multidimensional stream data has a data portion 
having a plurality of channels and a header portion 
describing a stream data format of the data portion 
and, 

20 

each of the plurality of channels has a plurality 
of pieces of information regarding a three- 
dimensional CG character at a certain time. 

16. A transmitter according to claim 15, wherein the 25 
three-dimensional CG character is capable of being 
represented by a hierarchical structure having at 
least one joint, and 

each of the plurality of motion data has at least 30 
one of data having a dimension of a position 
vector at a reference position of the three- 
dimensional CG character, data having a 
dimension of a direction vector representing a 
posture at the reference position, data having a 35 
dimension of an angle vector of the at least one 
joint and a rotation angle around the angle vec- 
tor, and stream data having a dimension of a 
rotation angle around a local coordinates axis 
of the at least one joint. 40 

17. A transmitter according to claim 1, wherein the 
multi-dimensional stream data is a packet having a 
header portion and a data portion, and 

45 

the header portion represents a kind of the 
packet, an order of the packet, and a length of 
the data portion. 

18. A transmitter according to claim 2, comprising a so 
multi-dimensional stream data retaining unit for 
retaining data compressed by the multi-dimen- 
sional stream data compressing unit in a multi-hier- 
archical structure. 

wherein the transmitter selects data retained ss 
in a certain hierarchical structure from the data 
retained in the multi-hierarchical structure, in 
accordance with a state of a transmission path 



through which the multi-dimensional stream data is 
to be transmitted and a reproduction speed at 
which a receiver receiving the multi-dimensional 
stream data reproduces the multi-dimensional 
stream data, and transmits the selected data. 

19. A transmitter according to claim 18, which selects 
data at a hierarchy level, at which a reproducible 
data amount is the largest, from the data retained in 
the multi-hierarchical structure, in accordance with 
a state of a transmission path through which the 
multi-dimensional stream data is to be transmitted, 
and a reproduction speed at which the receiver 
receiving the multi-dimensional stream data repro- 
duces the multi-dimensional stream data. 

20. A transmitter according to claim 2, further compris- 
ing a multi-dimensional stream data retaining unit 
for retaining data compressed by the multi-dimen- 
sional stream data compressing unit in a file format 
of the multi-dimensional stream data. 

21. A transmitting method comprising a multi-dimen- 
sional stream data transmitting step of transmitting 
multi-dimensional stream data. 

22. A transmitting method according to claim 21 , com- 
prising: 

a multi-dimensional stream data identifying 
step of classifying input data into a plurality of 
kinds of motion data; and 
a multi-dimensional stream data compressing 
step of compressing the plurality of kinds of 
motion data on a block basis. 

23. A transmitting method according to claim 22, 
wherein the multi-dimensional stream data com- 
pressing unit changes a compression ratio in 
accordance with the kind of the motion data. 

24. A transmitting method according to claim 22, 
wherein the multi-dimensional stream data com- 
pressing unit selects a compression method in 
accordance with the kind of the motion data. 

25. A transmitting method according to claim 22, 
wherein in the multi-dimensional stream data com- 
pressing step, a node sequence is generated, 
which is time-series data, composed of at least one 
motion data among the plurality of kinds of motion 
data during a predetermined period, and extracts 
an arbitrary node from the node sequence to com- 
press the plurality of kinds of motion data on a block 
basis. 

26. A transmitting method according to claim 25, com- 
prising the step of transmitting interpolation infor- 
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mation required for reproducing the extracted node 
and the node sequence. 

27. A transmitting method according to claim 22, 
wherein in the multi-dimensional stream data com- 
pressing step, a high frequency component of the 
plurality of kinds of motion data is reduced to com- 
press the plurality of kinds of motion data on a block 
basis. 

28. A transmitting method according to claim 22, 
wherein in the multi-dimensional stream data com- 
pressing step, at least one motion data among the 
plurality ot kinds of motion data is subjected to coor- 
dinate transformation to compress the plurality of 
kinds of motion data on a block basis. 

29. A transmitting method according to claim 22, 
wherein in the multi-dimensional stream data com- 
pressing step, data represented by three-dimen- 
sional vector coordinates among the plurality of 
kinds of motion data is transformed to data repre- 
sented by polar coordinates, thereby compressing 
the plurality of kinds of motion data on a block 
basis. 

30. A transmitting method according to claim 21 , 
wherein the multi-dimensional stream data includes 
header information or stream format describing 
data, describing a kind of the multi-dimensional 
stream data to be transmitted, an order of the plu- 
rality of kinds of motion data to be transmitted, and 
a length of the plurality of kinds of motion data to be 
transmitted. 

31. A transmitting method according to claim 21, 
wherein the multi-dimensional stream data has time 
information and space information of a plurality of 
dimensions. 

32. A transmitting method according to claim 31, 
wherein the space information of a plurality of 
dimensions includes position, information, rotation 
information, and scale information. 

33. A transmitting method according to claim 21, 
wherein the multi-dimensional stream data capable 
of representing a CG scene includes data repre- 
senting a position, rotation and a scale of the CG 
scene. 

34. A transmitting method according to claim 21, 
wherein the multi-dimensional stream data capable 
of representing a CG scene is multi-dimensional 
animation stream data. 

35. A transmitting method according to claim 21, 
wherein the multi-dimensional stream data has a 



data portion having a plurality of channels, and a 
header portion describing a stream data format of 
the data portion, and 

5 each of the plurality of channels has a plurality 

of pieces of information regarding a three- 
dimensional CG character at a certain time. 

36. A transmitting method according to claim 35, 
w wherein the three-dimensional CG character is 

capable of being represented by a hierarchical 
structure having at least one joint, and 

each of the plurality of motion data has at least 
is one of data having a dimension of a position 

vector at a reference position of the three- 
dimensional CG character, data having a 
dimension of a direction vector representing a 
posture at the reference position, data having a 
20 dimension of an angle vector of the at least one 

joint and a rotation angle around the angle vec- 
tor, and stream data having a dimension of a 
rotation angle around a local coordinates axis 
of the plurality of joints. 

25 

37. A transmitting method according to claim 21, 
wherein the multi-dimensional stream data is a 
packet having a header portion and a data portion, 
and 

30 

the header portion represents a kind of the 
packet, an order of the packet, and a length of 
the data portion. 

35 38. A transmitting method according to claim 22, com- 
prising a multi-dimensional stream data retaining 
step of retaining data compressed by the multi- 
dimensional stream data compressing unit in a 
multi-hierarchical structure, 

40 wherein the transmitting method selects 

data retained in a certain hierarchical structure from 
the data retained in the multi -hierarchical structure, 
in accordance with a state of a transmission path 
through which the multi -dimensional stream data is 

45 to be transmitted and a reproduction speed for 
reproducing the multi -dimensional stream data, and 
transmits the selected data. 

39. A transmitting method according to claim 38, 
so wherein data at a hierarchy level, at which a repro- 
ducible data amount is the largest, is selected from 
the data retained in the multi-hierarchical structure, 
in accordance with a state of a transmission path 
through which the multi-dimensional stream data is 
55 to be transmitted, and a reproduction speed at 
which the receiver receiving the multi-dimensional 
stream data reproduces the multi-dimensional 
stream data. 
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40. A transmitting method according to claim 22, fur- 
ther comprising a mufti-dimensional stream data 
retaining step of retaining data compressed in the 
multi-dimensional stream data compressing step in 
a file format of the multi-dimensional stream data. 

41. A receiver comprising a multi-dimensional stream 
data receiving unit for receiving multi-dimensional 
stream data. 

42. A receiver according to claim 41 , comprising: 

a multi-dimensional stream data identifying unit 
for identifying a data content of the multi- 
dimensional stream data from one of header 
information and stream format describing infor- 
mation of the multi-dimensional stream data; 
a multi-buffering unit for sequentially reading 
the multi-dimensional stream data into a plural- 
ity of multi-buffers on a block basis; and 
a multi-dimensional stream data decompress- 
ing unit for decompressing the multi-dimen- 
sional put in blocks on a block basis. 

43. A receiver according to claim 42, wherein the multi- 
dimensional stream data identifying unit deter- 
mines whether or not the received multi-dimen- 
sional stream data is a channel definition packet. 

44. A receiver according to claim 42, wherein the multi- 
dimensional stream data identifying unit deter- 
mines whether or not the received multi-dimen- 
sional stream data is a data packet. 

45. A receiver according to claim 43, wherein in the 
case where the multi-dimensional stream data 
identifying unit determines that the received multi- 
dimensional stream data is a channel definition 
packet, the multi-dimensional stream data identify- 
ing unit identifies a time stamp, a total number of 
channels, and a compression ratio identifier repre- 
senting a compression ratio of channel data from 
the received multi -dimensional stream data. 

46. A receiver according to claim 44, wherein in the 
case where the multi -dimensional stream data 
identifying unit determines that the received multi- 
dimensional stream data is a data packet, the multi- 
dimensional stream data identifying unit identifies a 
time stamp and a total number of channels from the 
received multi-dimensional stream data. 



dimensional stream data decompressing unit 
decompresses the compressed motion data based 
on the compression information. 

s 49. A receiver according to claim 48, wherein the multi- 
dimensional stream data decompressing unit deter- 
mines a decompression ratio based on the com- 
pression information. 

10 50. A receiver according to claim 48, wherein the multi- 
dimensional stream data decompressing unit deter- 
mines a decompression method based on the com- 
pression information. 

15 51. A receiver according to claim 48, wherein in the 
case where the compressed motion data is gener- 
ated by extracting arbitrary motion data from a plu- 
rality of motion data corresponding to a 
predetermined period, the multi-dimensional 

20 stream data decompressing unit recovers a plurality 
of motion data corresponding to the predetermined 
period based the extracted motion data and the 
compression information. 

25 52. A receiver according to claim 51 , wherein the com- 
pression information is interpolation information. 

53. A receiver according to claim 48, wherein the multi- 
dimensional stream data decompressing unit 
30 decompresses the compressed motion data by 
subjecting the compressed motion data to coordi- 
nate transformation based on the compression 
information. 

35 54. A receiver according to claim 42, wherein in the 
case where the compressed motion data is repre- 
sented by polar coordinates, the multi-dimensional 
stream data decompressing unit decompresses the 
compressed motion data by converting the com- 

40 pressed motion data represented by polar coordi- 
nates into data represented by three-dimensional 
vector coordinates. 

55. A receiver according to claim 41, wherein the multi- 
45 dimensional stream data has time information and 

space information of a plurality of dimensions. 

56. A receiver according to claim 55, wherein the space 
information of a plurality of dimensions has position 

so information, rotation information, and scale infor- 
mation. 



47. A receiver according to claim 41, wherein the 
received multi-dimensional stream data contains 
compressed motion data and compression informa- 
tion. 

48. A receiver according to claim 47, wherein the mutti- 



57. A receiver according to claim 41 , wherein the multi- 
dimensional stream data capable of representing a 

55 CG scene has data representing a position, a rota- 
tion, and a scale of the CG scene. 

58. A receiver according to claim 41 , wherein the multi- 
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dimensional stream data capable of representing a 
CG scene is multi-dimensional animation stream 
data. 

59. A receiver according to claim 41 , wherein the multi- 
dimensional stream data has a data portion having 
a plurality of channels and a header portion 
describing a stream data format of the data portion, 
and 

each of the plurality of channels has a plurality 
of pieces of information regarding a three- 
dimensional CG character at a certain time. 

60. A receiver according to claim 59, wherein the three- 
dimensional CG character is capable of being rep- 
resented by a hierarchical structure having at least 
one joint, and 

each of the plurality of motion data has at least 
one of data having a dimension of a position 
vector at a reference position of the three- 
dimensional CG character, data having a 
dimension of a direction vector representing a 
posture at the reference position, data having a 
dimension of an angle vector of the at least one 
joint and a rotation angle around the angle vec- 
tor, and stream data having a dimension of a 
rotation angle around a local coordinates axis 
of the plurality of joints. 

61 . A receiver according to claim 41 , wherein the multi- 
dimensional stream data is a packet having a 
header portion and a data portion, and 

the header portion represents a kind of the 
packet, an order of the packet, and a length of 
the data portion. 

62. A receiver according to claim 41, wherein the 
receiver indicates a compression ratio of the multi- 
dimensional stream data to a transmitter for trans- 
mitting the multi-dimensional stream data, in 
accordance with at least one of a state of a trans- 
mission path through which the multi-dimensional 
stream data is to be transmitted and a reproduction 
speed at which the receiver reproduces the multi- 
dimensional stream data. 

63. A receiving method comprising a multi-dimensional 
stream data receiving step of receiving multi- 
dimensional stream data. 

64. A receiving method according to claim 63, compris- 
ing: 

a multi-dimensional stream data identifying 
step of identifying a data content of the multi- 



dimensional stream data from one of header 
information and stream format describing infor- 
mation of the multi-dimensional stream data; 
a multi-buffering step of sequentially reading 

s the multi-dimensional stream data into a plural- 

ity of multi-buffers on a block basis; and 
a multi-dimensional stream data decompress- 
ing step of decompressing the multidimen- 
sional stream data put in blocks on a block 

w basis. 

65. A receiving method according to claim 64, wherein 
in the multi-dimensional stream data identifying 
step, it is determined whether or not the received 

75 multi-dimensional stream data is a channel defini- 
tion packet. 

66. A receiving method according to claim 64, wherein 
in the multi-dimensional stream data identifying 

20 step, it is determined whether or not the received 
multi-dimensional stream data is a data packet. 

67. A receiving method according to claim 65, wherein 
in the case where the multi-dimensional stream 

25 data identifying unit determines that the received 
multi-dimensional stream data is a channel defini- 
tion packet, in the multi-dimensional stream data 
identifying step, a time stamp, a total number of 
channels, and a compression ratio identifier repre- 

30 senting a compression ratio of channel data are 
identified from the received multi -dimensional 
stream data. 

68. A receiving method according to claim 66, wherein 
35 in the case where the multi-dimensional stream 

data identifying unit determines that the received 
multi-dimensional stream data is a data packet, in 
the multi-dimensional stream data identifying step, 
a time stamp and a total number of channels are 
40 identified from the received multi -dimensional 
stream data. 

69. A receiving method according to claim 63, wherein 
the received multi -dimensional stream data con- 

45 tains compressed motion data and compression 
information. 

70. A receiving method according to claim 69, wherein 
the multi-dimensional stream data decompressing 

so unit decompresses the compressed motion data 
based on the compression information. 

71. A receiving method according to claim 70, wherein 
in the multi -dimensional stream data decompress- 

55 ing step, a decompression ratio is determined 
based on the compression information. 

72. A receiving method according to claim 70, wherein 
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in the multi-dimensional stream data decompress- 
ing step, a decompression method is determined 
based on the compression information. 

73. A receiving method according to claim 70, wherein 
in the case where the compressed motion data is 
generated by extracting arbitrary motion data from 
a plurality of motion data corresponding to a prede- 
termined period, in the multi-dimensional stream 
data decompressing step, a plurality of motion data 
corresponding to the predetermined period are 
recovered based on the extracted motion data and 
the compression information. 

74. A receiving method according to claim 73, wherein 
the compression information is interpolation infor- 
mation. 

75. A receiving method according to claim 70, wherein 
in the multi-dimensional stream data decompress- 
ing step, the compressed motion data is decom- 
pressed by subjecting the compressed motion data 
to coordinate transformation based on the com- 
pression information. 

76. A receiving method according to claim 64, wherein 
in the case where the compressed motion data is 
represented by polar coordinates, in the multi- 
dimensional stream data decompressing step, the 
compressed motion data is decompressed by con- 
verting the compressed motion data represented by 
polar coordinates into data represented by three- 
dimensional vector coordinates. 

77. A receiving method according to claim 63, wherein 
the multi-dimensional stream data has time infor- 
mation and space information of a plurality of 
dimensions. 

78. A receiving method according to claim 77, wherein 
the space information of a plurality of dimensions 
has position information, rotation information, and 
scale information. 

79. A receiving method according to claim 63, wherein 
the mutti -dimensional stream data capable of repre- 
senting a CG scene has data representing a posi- 
tion, a rotation, and a scale of the CG scene. 

80. A receiving method according to claim 63, wherein 
the multi-dimensional stream data capable of repre- 
senting a CG scene is muiti -dimensional animation 
stream data. 

81. A receiving method according to claim 63, wherein 
the multi-dimensional stream data has a data por- 
tion having a plurality of channels and a header por- 
tion describing a stream data format of the data 



portion and, 

each of the plurality of channels has a plurality 
of pieces of information regarding a three- 
s dimensional CG character at a certain time. 

82. A receiving method according to claim 81 , wherein 
the three-dimensional CG character is capable of 
being represented by a hierarchical structure hav- 

10 ing at least one joint, and 

each of the plurality of motion data has at least 
one of data having a dimension of a position 
vector at a reference position of the three- 

15 dimensional CG character, data having a 

dimension of a direction vector representing a 
posture at the reference position, data having a 
dimension of an angle vector of the at least one 
joint and a rotation angle around the angle vec- 

20 tor, and stream data having a dimension of a 

rotation angle around a local coordinates axis 
of the at least one joint. 

83. A receiving method according to claim 63, wherein 
25 the multi-dimensional stream data is a packet hav- 
ing a header portion and a data portion, and 

the header portion represents a kind of the 
packet, an order of the packet, and a length of 
30 the data portion. 

84. A receiving method according to claim 63, wherein 
a compression ratio of the multi-dimensional 
stream data is indicated to a transmitter for trans- 

35 mitting the multi-dimensional stream data, in 
accordance with at least one of a state of a trans- 
mission path through which the multi-dimensional 
stream data is to be transmitted and a reproduction 
speed for reproducing the multi-dimensional stream 

40 data. 

85. A receiver according to claim 41 , wherein the multi- 
dimensional stream data contains motion data and 
voice data, 

45 

the motion data has time stamp information, 
and the voice data has time stamp information, 
and 

the receiver reproduces the motion data and 
so the voice data in synchronization, in accord- 

ance with the time stamp information of the 
motion data and the time stamp information of 
the voice data. 

55 86. A receiving method according to claim 63, wherein 
the multi -dimensional stream data contains motion 
data and voice data, 
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the motion data has time stamp information, 
and the voice data has time stamp information, 
and 

the receiving method further comprises the 
step of reproducing the motion data and the 5 
voice data in synchronization, in accordance 
with the time stamp information of the motion 
data and the time stamp information of the 
voice data. 

10 

87. A recording medium capable of being read by a 
computer, in which a program for allowing the com- 
puter to execute the transmitting method of claim 2 1 
is recorded. 

15 

88. A recording medium capable of being read by a 
computer, in which a program for allowing the com- 
puter to execute the receiving method of claim 63 is 
recorded. 

20 

89. A recording medium capable of being read by a 
computer, in which a program for allowing the com- 
puter to execute the receiving method of claim 86 is 
recorded. 

25 

90. A communication system comprising: 



a transmitter for transmitting multi-dimensional 
stream data; and 

a receiver for receiving the multi-dimensional 30 
stream data. 

91. A communication system according to claim 90, 
wherein the transmitter includes one of an artificial 
satellite, a ground wave broadcasting station, and a 35 
wire broadcasting station, and the receiver is one of 
a digital TV and a set top box. 



92. A transmitter according to claim 1, wherein the 
multi-dimensional stream data contains data capa- 40 
ble of representing a CG scene. 

93. A transmitting method according to claim 21, 
wherein the multi-dimensional stream data contains 
data capable of representing a CG scene. 45 

94. A receiver according to claim 41 , wherein the multi- 
dimensional stream data contains data capable of 
representing a CG scene. 

50 

95. A receiving method according to claim 63, wherein 
the multi-dimensional stream data contains data 
capable of representing a CG scene. 

96. A communication system according to claim 90, ss 
wherein the multi-dimensional stream data contains 
data capable of representing a CG scene. 
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